Statistical Methods for Bullet Matching

Eric Riemer Hare

Statistical Methods for Bullet Matching

Eric Hare
Iowa State University
April 5th, 2017

Background

About Me

Publications

Awards

Dissertation Outline

Statistical Methods for Bullet Matching

Papers:

  1. Automatic Matching of Bullet Lands
  2. Bullet Land Feature Analysis
  3. A Modern Bullet Matching Application

Dissertation Chapter One

Automatic Matching of Bullet Lands

Eric Hare, Heike Hofmann, Alicia Carriquiry
Center for Statistics and Applications in Forensic Evidence (CSAFE)

Goal

Current Practice

The problems culminated in a 2009 NAS report which found “much forensic evidence – including, for example, bite marks and firearm and toolmark identification is introduced in criminal trials without any meaningful scientific validation, determination of error rates, or reliability testing.” (National Research Council 2009)

PCAST Report

From a September 2016 report by the President’s Council of Advisors on Science and Technology (PCAST) titled Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (Advisors on Science and Technology 2016):

A second—and more important—direction is (as with latent print analysis) 
to convert firearms analysis from a subjective method to an objective 
method. This would involve developing and testing image-analysis 
algorithms for comparing the similarity of tool marks on bullets. [...] 
In a recent study, researchers used images from an earlier study to 
develop a computer-assisted approach to match bullets that minimizes 
human input [338].

338: Hare, E., Hofmann, H., and A. Carriquiry. “Automatic matching of bullet lands.” Unpublished paper, available at: arxiv.org/pdf/1601.05788v2.pdf.

Our Approach

The key to this approach is the reference database

James Hamby Study

plot3D.x3p.file(read_x3p("../images/Hamby (2009) Barrel/bullets/Br1 Bullet 1-5.x3p"),
plot.type = "surface")

Data Format

Step One: Extract a Profile

We need to choose a location (height) of the bullet at which to extract a profile. To do so, we optimize the CCF (T. Vorburger et al. 2011):

  1. Extract a profile near the base of the bullet, call this value d0.
  2. Take a fixed step d and extract at d + d0.
  3. Check the maximum cross correlation (CCF) between the signature at d0 and at d + d0.
  4. If this CCF exceeds a threshold c, choose d0 as the signature
  5. Otherwise, repeat steps 2 to 4 for d, 2d, 3d, … until the threshold is achieved.
  6. If the threshold is not achieved, flag the land for further investigation.

Parameters: d = 25μm, d0 = 25μm, c = 0.9

Step One (Continued)

br111 <- get_crosscut("images/Br1 Bullet 1-5.x3p", x = 243.75)

qplot(y, value, data = br111) + theme_bw()

Step Two: Remove Shoulders

The striations that identify a bullet to a gun barrel are located in the land impression areas (Xie et al. 2009).

  1. At a fixed height x extract a bullet’s profile (previous figure, with x = 243.75μm).
  2. For each y value, smooth out any deviations occurring near the minima by applying a rolling average with a pre-set s.
  3. For each smoothed y value, compute another rolling average using the same smoothing factor s as above.
  4. Determine the location of the peak of the shoulders by finding the first and last doubly-smoothed value yi that is the maximum within its smoothing window.

Parameters: s = 35μm

Identifying Shoulders (Easy)

br111.groove <- get_grooves(br111)
br111.groove$plot

Identifying Shoulders (Challenging)

result2 <- get_grooves(get_crosscut("../images/Hamby (2009) Barrel/bullets/Br1 Bullet 1-6.x3p"))
result2$plot

Step Three: Fit Loess Regression

Local weighted scatterplot smoothing (Cleveland 1979) - Fits a low-degree polynomial to a small subset of the data, weighting values near the point to be estimated more strongly.

br111.loess <- fit_loess(br111, br111.groove)
br111.loess$fitted

Step Four: Get the Residuals

Deviations from the loess fit should represent the imperfections (striations) on the bullet. Hence, we extract the residuals from the model.

br111.loess$resid

Step Five: Peaks and Valleys

As with detecting the shoulders, we can smooth the deviations and compute derivatives to identify peaks and valleys in the signature.

br111.peaks <- get_peaks(br111.loess$data)
br111.peaks$plot

Step Six: Bullet Alignment

The previous five steps are performed for each bullet land. But now we wish to extract features for cross comparisons of bullet lands.

Step Six (Continued)

Step Six: Extract Features

Features are extracted from each land-to-land comparison:

More Features

Distribution of Features

Step Seven: Random Forest

Feature Importance

Previous Future Work

Verbatim from September:

We have steps to address each of these concerns…

Data Limitation

NIST has provided some more data:

  1. Hamby (Set 44) - These are the same barrels that were used for the Hamby study, but the bullets were scanned by a different person which allows us to provide an initial assessment of operator or scan quality effects
  2. Cary Persistence - Repeated test fires from a single gun barrel
  3. Our own microscope - We have installed a Sensofar 3D imaging microscope at Iowa State, and will soon begin a controlled study with numerous barrels and bullets.

Feature Standardization

To begin to tackle the degraded bullet problem, we need to standardize features by the length of the recovered land.

Matches = 27, Matches per mm = 14.72

True Degraded Case

By standardizing the features, we don’t penalize the degraded case as in the first revision of our algorithm:

Matches = 8, Matches per mm = 11.42

Degraded Bullets

Simulation Study:

  1. Three types of Degradation:
    1. Left Fixed - The left portion of the land (leading shoulder) is recoverable
    2. Middle Fixed - The middle portion of the land is recoverable
    3. Right Fixed - The right portion of the land (trailing shoulder) is recoverable
  2. Six Degradation Levels: 100% (Fully recovered), 87.5% Recovered, 75% Recovered, 62.5% Recovered, 50% Recovered, 25% Recovered

Simulation Caveats

We simulated the degradation of the processed signatures rather than simulating a degraded surface.

This means that the recovered signature may have been slightly influenced by points that were removed in the simulation, though this effect is likely to be minor

Simulation Results

Feature Expression

Testing this Finding

This confirms a NIST theory that the leading shoulder of the bullet better expresses striations as it travels through the barrel. To further test this, we can attempt to match a previously excluded land (due to severe tank rash) to its known match.

Br924 Results

Extracting the ideal signature and then simulating a left-fixed 50% degradation scenario yields the following:

Operator Effects

With two studies, Hamby 252 and Hamby 44, there are three sets of cross comparisons we perform:

  1. Hamby 252 to Hamby 252
  2. Hamby 252 to Hamby 44
  3. Hamby 44 to Hamby 44

Given that the barrels are the same in each case, assuming perfect scans and no microscope operator effects, each set of comparisons should be indistinguishable from one another.

Feature Expression by Study

Ideal Cross-Section by Study

From Lands to Bullets

In a real world forensic application of these algorithms, the true experimental unit is the bullet rather than the land

Bullet Method 1

Idea: Two bullets b1 and b2 are matches if and only if some land lb1 from b1 matches some land lb2 from bullet 2. Corollary: Two bullets are non-matches if and only if no land from b1 matches a land from b2. Assumptions: Each land on a bullet is independent of each other land


$$ \begin{align} P(M) &= 1 - P(NM) \\ &= 1 - (P(NM1) \times P(NM2) \times ... \times P(NM6)) \\ &= 1 - ((1 - P(M1)) \times (1 - P(M2)) \times ... \times (1 - P(M6))) \end{align} $$

Exploiting the Alignment

The true alignment of the lands exists along one of 6 diagonals:

Therefore, we can multiply the probabilities across the diagonals and take the highest for each bullet to bullet comparison (Sensorfar 2017)

Bullet to Bullet Results

Bullet Method 2

Assuming bullets match if and only if all lands match:

Performance is weaker in this case because our algorithm is much more likely to make false negatives compared to false positives.

Software Design Principles

In Designing Modular Software: A Case Study in Introductory Statistics, we outlined several software design principles:

  1. Modularity
  2. Reproducibility
  3. Extensibility
  4. Residing on the Web

To truly open these bullet matching methods to the scientific community, we had to focus on a system which adheres to these principles

Database

The underlying technology we’ve created since September is a brand new database which modularizes the algorithm and allows collaboration on individual components. The database:

  1. Stores all parameters used in each run of the algorithm
  2. Keeps track of intermediate results (profiles, signatures, etc.)
  3. Is structured to allow seamless running and re-running of the algorithm
  4. Allows researchers to extract the relevant components for their own research

Structure

Front-End Web Application

https://isu-csafe.stat.iastate.edu/shiny/bulletr/

Back-End Web Application

https://isu-csafe.stat.iastate.edu/shiny/bullets/

Thank You

Special thanks to Alan Zheng at the National Institute of Standards and Technology for maintaining the NIST Ballistics Toolmark Research Database and providing many useful suggestions for our algorithm.

Any Questions?

Bibliography

Advisors on Science, President’s Council of, and Technology. 2016. “Report on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods.” https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_forensic_science_report_final.pdf.

Biasotti, Alfred A. 1959. “A Statistical Study of the Individual Characteristics of Fired Bullets.” Journal of Forensic Sciences 4 (1): 34–50.

Chu, Wei, Robert M Thompson, John Song, and Theodore V Vorburger. 2013. “Automatic identification of bullet signatures based on consecutive matching striae (CMS) criteria.” Forensic Science International 231 (1–3): 137–41.

Clarkson, James A, and C Raymond Adams. 1933. “On Definitions of Bounded Variation for Functions of Two Variables.” Transactions of the American Mathematical Society 35 (4). JSTOR: 824–54.

Cleveland, William S. 1979. “Robust Locally Weighted Regression and Smoothing Scatterplots.” Journal of the American Statistical Association 74 (368). Taylor & Francis, Ltd.: 829–36. http://www.jstor.org/stable/2286407.

Giannelli, Paul C. 2011. “Ballistics Evidence Under Fire.” Criminal Justice 25 (4): 50–51.

Hamby, James E., David J. Brundage, and James W. Thorpe. 2009. “The Identification of Bullets Fired from 10 Consecutively Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries.” AFTE Journal 41 (2): 99–110.

Hofmann, Heike, and Eric Hare. 2016. Bulletr: Algorithms for Matching Bullet Lands.

National Research Council. 2009. Strengthening Forensic Science in the United States: A Path Forward. Washington, DC: The National Academies Press. doi:10.17226/12589.

Nichols, Ronald G. 2003. “Consecutive Matching Striations (CMS): Its Definition, Study and Application in the Discipline of Firearms and Tool Mark Identification.” AFTE Journal 35 (3): 298–306.

OpenFMC. 2014. X3pr: Read/Write Functionality for X3p Surface Metrology Format.

Sensorfar. 2017. SensoMATCH Bullet Comparison Software.

Vorburger, T.V., J.-F. Song, W. Chu, L. Ma, S.H. Bui, A. Zheng, and T.B. Renegar. 2011. “Applications of Cross-Correlation Functions.” Wear 271 (3–4): 529–33. doi:http://dx.doi.org/10.1016/j.wear.2010.03.030.

Xie, F., S. Xiao, L. Blunt, W. Zeng, and X. Jiang. 2009. “Automated Bullet-Identification System Based on Surface Topography Techniques.” Wear 266 (5–6): 518–22. doi:http://dx.doi.org/10.1016/j.wear.2008.04.081.